Read images with deepseek-v4-flash-vision-exp, and move DeepSeek onto the Responses protocol - #185
Merged
Merged
Conversation
DeepSeek documents deepseek-v4-flash-vision-exp and its image input only outside the pages the snapshot already carried, so llmsdk_docs/deepseek_v4/ gains docs/vision.md (the three input methods per protocol, detail levels, size and count limits, and the 384-token upper bound per image) and docs/responses-api.md (stream event names, parameter and input-item coverage, usage fields). The quickstart lists the third model and the 1M/384K context figures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
DeepSeekV4Client now calls /responses instead of /chat/completions: it reads the reasoning_text, output_text and function_call_arguments deltas, rebuilds a replayed chain of thought as a reasoning item whose content is reasoning_text, and takes usage from input_tokens/output_tokens with their detail blocks. thinking_level maps to reasoning.effort pre-collapsed to what DeepSeek settles on (none/low/high/high/high/max); effort "none" is what disables thinking there, since the Chat Completions thinking toggle is ignored on this endpoint. The temperature, tool_choice, fast_mode and prompt_caching restrictions are unchanged. An image_url item is sent as an input_image part and a tool result's images ride in the function_call_output, so a tool can hand the model a picture it produced. A model without vision refuses the item instead: Chat Completions answers 400, but the Responses API silently swaps the image for placeholder text. Both this client and OpenaiResponsesClient now flush the message text collected so far before appending a reasoning, function_call or function_call_output item. An assistant turn that spoke before calling a tool used to replay as call, then text, then output, which DeepSeek answered with "No tool output found for tool call <id>" on the next turn. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
The four protocol suites fed DeepSeek chat-completion chunks through a fake .chat.completions endpoint, which the Responses client no longer calls. The DeepSeek case moves to the Responses shape in each: the stream-case lists carry a protocol field, the fakes install either endpoint, and the empty-response and tool-call-argument suites gained a case for the generic openai-responses client beside it. The thinking-level table now expects effort "none" for NONE, which is how thinking is disabled on this endpoint. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
…-chat The vision model joins the registry on the official API with Text, Image input, a 1M context window and the deepseek-v4-flash price (CNY 1.5 / 4.5 per million tokens off-peak, 0.05 on a cache hit; peak rates are double). It replaces deepseek-v4-flash as the official DeepSeek entry in both E2E model lists with image understanding enabled, which brings the URL, base64 and tool-result image tests with it; the three protocol-mode entries stay on deepseek-v4-flash. deepseek-ai/DeepSeek-V4-Flash and deepseek-ai/DeepSeek-V4-Pro move to the generic openai-chat client, since api.siliconflow.cn serves no /responses endpoint. The OpenRouter entries keep the deepseek-v4 client, which its /responses does serve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
The README table names what each client_type actually speaks: google-genai for the Gemini family, ant-messages for Claude, openai-responses for GPT, DeepSeek and MiniMax, openai-chat for GLM and Kimi, and the OpenAI Embeddings endpoint for openai-embedding. The generic protocol clients name their own protocol. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
…ocol section The Responses transforms decide once, at the top of the item loop, that anything which is not message content flushes the text collected so far; the branch chain below is the one it had before. The continue-based split said the same thing in more places. In the README the three generic protocol clients become a list, the supported-model listing gets a Python and a TypeScript snippet, the client-type-to-protocol table moves below both, and openai-embedding is code-formatted like the other protocols. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
…h tests GPT5_6Client and MiniMaxM3Client deferred the assistant message to the end of the item loop the same way the other two Responses clients did, so an assistant turn that spoke before calling a tool replayed as call, then text. Both now flush the collected text before appending a standalone input item. The audit of the remaining clients found nothing else: Claude and the Anthropic protocol client build one ordered content block list, Gemini one ordered parts list, and Chat Completions carries no interleaving at all, since the text lands in content and the call in tool_calls of the same message. test_message_order.py and message-order.test.ts run a thinking-text-tool_call turn through all ten clients with a fake key and assert the emitted order per protocol. Against the pre-fix tree the four Responses cases fail and the other six pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
…down An "unused" event is a client's own marker for a wire event that carries nothing universal. No client yields one today, but nothing guaranteed it: streaming_response now drops one that escapes and raises on it under AGENTHUB_DEBUG, so a client that forgets its filter fails in CI instead of shipping empty events to a caller. The EventType definition says what the four kinds mean, and the development skill records both client rules it had only in code: unused stays inside the client, and the message transform keeps the order of the content items. The unknown-event suites now run every client over the ignorable events its protocol carries with the debug guard on, and a deliberately leaky client covers the drop and the raise. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
deepseek-v4-flash-vision-expreads images, DeepSeek's dedicated client moves off ChatCompletions onto the Responses API, and a replay-ordering bug that both Responses clients
carried is fixed.
What the live API does
Probed against
api.deepseek.comon 2026-08-21 with a picture whose only content is the wordBANANA, never named in the prompt:BANANAon both protocolsfunction_call_outputwithinput_image)BANANANO_IMAGE, so the image really is what it read400 This model does not support image. Responses: no error, the image becomes placeholder text and the model answers "Unsupported"reasoning.effort: "none"extra_body {"thinking": {"type": "disabled"}}/responses: still 148 reasoning tokenstool_choice: "required"or a named tool400 Thinking mode does not support this tool_choicetemperature0.0 and 2.0service_tier: "priority"api.siliconflow.cn/v1/responses404 page not foundopenrouter.ai/api/v1/responseswithdeepseek/deepseek-v4-flashReplay probes on the captured tool-call turn: the reasoning item can come back as
reasoning_textcontent, as a summary, empty, withencrypted_content, or not at all — everyshape is accepted with behavior intact, so the client rebuilds the minimal one from the
thinking text.
The ordering bug
test_tool_usefailed intermittently against DeepSeek with400 No tool output found for tool call <id>. It happens whenever the assistant speaks beforecalling a tool: the transform appended the assistant message after the
function_callitpreceded, and DeepSeek merges a call into the adjacent assistant message. Isolated probe:
user → reasoning → function_call → assistant text → function_call_output→ 400user → reasoning → assistant text → function_call → function_call_output→ worksBoth
DeepSeekV4ClientandOpenaiResponsesClientnow flush collected message text beforeappending a standalone input item. The generic client carried the same latent bug.
Decisions taken (confirmed in the conversation)
/responses, so its two DeepSeek entries move to the genericopenai-chatclient rather than keeping a second DeepSeek client alive.none/low/high/high/high/max)instead of passing
medium/xhighthrough.UnsupportedParameterErrorfor a temperature other than 1.0 and for anything but
auto/none.degrades silently rather than failing.
shape; the empty-response and tool-call-argument suites also gained a case for the generic
openai-responsesclient, which had none.Also in this change
README.mdgained a table naming the wire protocol behind everyclient_type:google-genaifor the Gemini family,
ant-messagesfor Claude,openai-responsesfor GPT, DeepSeek andMiniMax,
openai-chatfor GLM and Kimi, and the OpenAI Embeddings endpoint foropenai-embedding.Verification
pytest -k deepseek35 passed (image URL, base64 image and tool-result image amongthem),
jest -t deepseek77 passed.pytest -k openai-responses9 passed across gateways, covering the shared ordering fix.ruff check/ruff format --check,npm run lintandnpm run buildclean.AutoLLMClient(test_list_supported_models).Not covered
deepseek-v4-flash-vision-expis an experimental model with no entry in the vendor's changelog;if it is withdrawn, the registry entry and the official E2E slot go with it.
🤖 Generated with Claude Code
https://claude.ai/code/session_01UBwEdjegVGtfY2yJKwhZ77